A dynamic programming approach to efficient sampling from Boltzmann distributions

Several algorithms in optimization, computer science and statistical mechanics are closely related to the problem of sampling from a Boltzmann distribution parameterized by the so-called temperature over a finite set. These include (i) Simulated Annealing (SA) [6], (ii) calculating the permanent of a nonnegative matrix [1], (iii) estimating the volume of a convex body [7], and (iv) computing partition functions of interacting particle systems such as the Ising model [8]. The finite set involved is typically exceedingly large rendering exact Boltzmann-sampling impractical. Hence one approach is to simulate “several” transitions of an ergodic Markov chain with the appropriate Boltzmann as its limiting distribution so that the final state-distribution wellapproximates this Boltzmann. More precisely, to sample from a Boltzmann distribution at temperature T∗ > 0, it is common to simulate a sequence of ergodic Markov chains whose limiting distributions are also Boltzmann at temperatures given by a “cooling schedule”—a strictly decreasing finite sequence of temperatures starting at a very high value, commonly ∞, and ending at T∗. Initial motivation for implementing the sampling procedure in phases defined by a cooling schedule came from annealing processes in physics where a glass or metal is toughened by cooling it slowly, starting at a high temperature, to a low temperature equilibrium. The mathematical intuition behind gradual cooling is to implement phases such that the distribution of the state at the end of one phase is not very different from the limiting distribution of the next phase with the hope that this will reduce the total number of iterations required to well-approximate the target Boltzmann. This concept is termed a “warm start” and has been featured in recent work in this area [5]. A search for an “optimal” cooling strategy must consider several key questions including how to select the number of Markov transitions in each phase. Specifically, given a fixed number of total iterations (this may typically arise from knowledge of available computational power, and computation time), running too many iterations in early phases may be wasteful, whereas too few may leave us “far away” from the limiting distributions of subsequent phases (“cold start”).